In recent years, studies of Mixed Reality (MR) for the purpose of seamless merging of a real space and virtual space have been extensively made. Conventionally, MR has received a lot of attention as a technique that aims at achieving coexistence of the real space and a virtual reality (VR) world that can be experienced in only a situation isolated from the real space, and augments VR.
A typical apparatus for implementing MR is a head-mounted display (HMD). That is, MR is implemented by displaying a composite image of the real space (or its video image) and virtual space image on the HMD. MR systems using the HMD includes an optical see-through system in which the user experiences an MR space by displaying a virtual space image generated by, e.g., computer graphics (CG) on a semi-transparent (see-through) head-mounted display (HMD) to composite by the user's eyes a real space that the user directly observes by his or her eyes and the virtual space image on the display, and a video see-through system in which the user experiences an MR space by generating an MR space image as a composite image of image data sensed by a video camera attached to, e.g., an HMD and a CG image or the like, and displaying the composite image on the HMD.
MR can be used, quite possibly, in new fields which are qualitatively quite different from VR, such as a medical assist application for presenting the state inside the body of a patient to a doctor as if it were seen through, a job assist application for superposing and displaying the assembly sequence of a product on real parts in a factory, a game application such as a shooting game, sports simulation, or the like that displays virtual enemies, ball, or the like generated by a virtual space image in a real space, a living simulation that allows the user to lay out virtual furniture in a physical space room, and the like.
In order to implement such MR space, the positional relationship of objects in the real space must be recognized. That is, when an object (virtual object) represented by a virtual space image is displayed, a portion occluded by an object present in the real space must be inhibited from being displayed. When a virtual object is moved, it must be inhibited from being moved to a position that overlaps an object in the real space. Also, when hitting determination between an object in the real space and virtual object is made in, e.g., a game, the position information of a real object is required.
Methods of acquiring object position information in the real space include a method using a laser rangefinder which scans a laser beam and obtains distance information (depth information) on the basis of the time until the returned laser beam is received, a multiview stereo algorithm that senses the real space using a plurality of video cameras, and obtains the distance from the image sensing position to an object on the basis of the sensed image data, and the like. On the other hand, the depth information is often used in the form of a depth map which is imaged by changing the luminance levels, assigning different colors, or the like in correspondence with different distances.
An MR presentation system must mix the real space and virtual space images at the user's viewpoint position without any disturbance, and present (display) the mixed image. Since the user's viewpoint position always moves, detection of depth information and viewpoint position information, and rendering based on such information must be done at a speed close to real time. Especially, in an application such as a game, since the user's viewpoint position moves quickly and largely, acquisition of depth information (generation of a depth map) requires high-speed processing.
However, in a conventional depth information measurement apparatus, if computations are made at high speed, the obtained depth information has low precision; if high-precision information is to be obtained, computations become complicated, and a long processing time is required.
The present invention has been made in consideration of the aforementioned conventional problems, and has as its object to provide a depth information measurement apparatus which can acquire high-precision depth information in a short computation time.